Clinical Trials

The efficacy of current antiretroviral interventions for prevention of vertical HIV transmission is both a triumph and a challenge for future efficacy studies of novel agents, including long-acting antiretroviral agents. Given the effectiveness of current antiretroviral interventions, standard randomized controlled trials to assess novel interventions targeted towards further reducing vertical HIV transmission may be infeasible due to required sample sizes. To address this study design challenge, we propose a novel augmented clinical trial design, where a randomized trial with an active comparator is augmented with a concurrent observational cohort to supplement the standard-of-care arm. The proposed design and analysis can overcome sampling biases, and an associated diagnostic can evaluate potential biases from unmeasured covariates. The proposed trial design is illustrated with an example and studied further using a simulation study. In the illustrative example, an augmented trial with a 3:1 allocation ratio in the trial component yielded an unbiased treatment effect estimate, as compared to the trial component alone, and the standard error was reduced substantially. Although the total sample size of the augmented trial was larger than a similarly powered 1:1 standard trial, the sample size of the augmented trial's trial component was half the size of the standard trial and within the range of historically similar trials, thereby providing both feasibility and potential cost advantages. Although limitations with this design remain, many of these limitations can be mitigated by careful study planning and coordination. Augmented trials provide a potentially applicable solution to studying the efficacy of long-acting antiretrovirals for infant postnatal prophylaxis against HIV.

Introduction: Risk of recurrent opioid use during buprenorphine-naloxone (bup-nx) treatment is dynamic and remains elevated after initiation, with vulnerability shaped in part by treatment intensity and gaps between visits, yet routine outpatient care relies on episodic encounters and retrospective data. This mismatch can delay recognition of emerging instability and limit timely treatment adjustments. This paper reports the development and specification of an intervention strategy to address this mismatch. Methods: We used a structured, multi-phase design process to specify and configure a measurement-based care (MBC) strategy for bup-nx treatment (Bup-MBC) in outpatient addiction clinics through three phases: (1) a systematic review of patient-reported outcome measures (PROMs) for substance use treatment; (2) a qualitative needs assessment using the Theoretical Domains Framework and COM-B (Capability, Opportunity, Motivation-Behavior) model to identify gaps in risk monitoring, agency, and trust; and (3) iterative co-design with multidisciplinary clinicians to refine workflow fit and trust-preserving use of data. Patients informed item and feedback content during the needs assessment but did not participate in the co-design cycles. Results: Bup-MBC integrates (1) brief between-visit PROMs (e.g., withdrawal, craving, adherence); (2) immediate non-punitive patient feedback; (3) clinician-facing summaries and non-directive prompts in the electronic health record (EHR); and (4) an opt-in between-visit outreach pathway with predefined safety triggers, all configured within existing EHR and patient portal infrastructure. It targets patient and clinician capability to recognize changes in risk, opportunity for action through structured monitoring and visit preparation, and trust and agency through non-punitive communication, without adding substantial burden. The full measure set, severity bands, and question-to-action map are provided as supplementary material. Key trade-offs included prioritizing single-item measures for feasibility, balancing opt-in outreach with safety overrides, and assuming routine clinician use of summaries. Conclusion: This development study specifies an EHR-integrated MBC strategy for outpatient bup-nx treatment. As single-center design work with co-design limited to clinicians and delivery contingent on portal or text-message access, its outputs are hypotheses about mechanism and fit rather than demonstrated effects. Feasibility studies are needed to evaluate uptake, acceptability, workflow fit, and effects on treatment.

TrialCalibre1 is an agentic system that connects clinical evidence, study design, and executable analyses. It supports reference-trial replication, direct target-trial emulation, and BenchExCal, which combines benchmarking, extension, and cal ibration sensitivity analysis. We evaluated an ASCET-based benchmark and an extension comparing clopidogrel with aspirin among adults aged 81-90 years us ing synthetic PCORnet-style claims. Two reviewers assessed benchmark extrac tion, protocol decisions, and implementation. After discussion, 53/62 and 50/62 assessable SQL checks were rated met; both reviewers rated 12/13 assessable R checks met. Protocol and proxy disagreements remained. A separate ten-question routing development set yielded 10/10 route matches and 7/7 applicable reference trial matches. The agent benchmark and extension included 6,069 and 1,602 pa tients. The extension risk ratio was 0.848 (95% CI 0.569-1.184), with a calibrated sensitivity estimate of 1.274 (0.439-3.700). The human implementation provided descriptive comparisons, not same-data validation. Dose proxies, unpopulated co variates, residual imbalance, implementation deviations, and ignored cross-cohort covariance limited interpretation. This case demonstrates a connected workflow whose protocols, code, and results can be reviewed.

Background: Digital reminders reduce outpatient no-shows, but the optimal timing and frequency of messages remain unclear, particularly in Latin American public health systems. We emulated a target trial to evaluate the comparative effectiveness of four WhatsApp reminder strategies on appointment absenteeism and patient-initiated cancellations. Methods: We analyzed administrative and electronic health-record data from the public health system of the Autonomous City of Buenos Aires, Argentina (June 2023-May 2024). Eligible individuals had scheduled an in-person outpatient appointment in one of 15 prioritized specialties at least 75 hours in advance and had a mobile phone on record. We compared four strategies: (1) dual reminders at ~72 and ~24 hours before the appointment; (2) a single reminder at ~72 hours; (3) a single reminder at ~24 hours; and (4) no reminders. The primary outcome was the proportion of no-shows by the end of follow-up. Secondary outcomes were the cumulative incidence of patient-initiated cancellations overall, within 12 hours of the appointment, and followed by rebooking. We emulated the target trial using a cloning-censoring-weighting approach to estimate per-protocol controlled direct effects, with inverse-probability weights to address time-varying confounding and selection bias. Cumulative incidence of secondary outcomes was estimated using weighted Kaplan-Meier curves. Three pre-specified sensitivity analyses and standardized mean differences assessed robustness and covariate balance. Results: A total of 475,214 first eligible person-appointments were included; baseline no-show risk in the control arm was 34.6%. All three active strategies reduced no-shows compared with no reminders. The single 24-hour reminder produced the largest reduction (Risk Ratio [RR] 0.76, 95% CI 0.72, 0.81; Risk Difference [RD] -8.21 percentage points [pp], 95% CI -9.68, -6.54), followed by the dual-reminder strategy (RR 0.80, 95% CI 0.79,0.81; RD -7.05 pp, 95% CI -7.41, -6.71) and the single 72-hour reminder (RR 0.91, 95% CI 0.84,0.99; RD -3.16 pp, 95% CI -5.69, -0.49). All active strategies increased patient-initiated cancellations relative to control, with the dual-reminder strategy producing the largest increase. Sensitivity analyses preserved the qualitative ranking of strategies across all specifications. Conclusions: In this large target trial emulation, a single just-in-time WhatsApp reminder sent ~24 hours before the appointment was as effective as a dual-reminder schedule in preventing no-shows and superior to a distal 72-hour reminder alone. Adding a second, distal reminder provided no measurable benefit for attendance but substantially increased patient-initiated cancellations, which may be operationally valuable when active slot reallocation is a goal. These findings support timing, rather than frequency, as the primary lever of digital-reminder effectiveness, and favor the deployment of a single proximal reminder as the default strategy in resource-constrained outpatient settings.

In 2027, the National Institutes of Health (NIH) will no longer support costs associated with clinical trials (CT) for career development (K) awards. NIH's CT definition encompasses many types of research, from early-stage pilot work through definitive studies. The types and sizes of trials K awardees conduct have not been described, which could inform ongoing policy changes relevant to the next generation of clinical trialists. We conducted a cross-sectional analysis of NIH RePORTER data to characterize the types and sizes of CTs supported by NIH K01, K08, and K23 awards funded in fiscal years 2024-2025 under a CT Required or CT Optional funding opportunity announcement (FOA), with non-missing abstract text. A rule-based classification scheme applied to free-text abstracts categorized each award into 5 mutually exclusive trial types: pilot/feasibility/proof-of-concept, efficacy/effectiveness, trial confirmed with type unclear, other (mechanistic, ancillary, or phase I/II), or no trial-specific language identified. Trial size was assessed among abstracts reporting an explicit, manually verified sample size. Of 1336 unique awards funded under CT Required (n=1051) or CT Optional (n=285) FOAs, 815 (61.0%) were classified as pilot, feasibility, or proof-of-concept studies, and 55 (4.1%) as efficacy or effectiveness studies; 204 (15.3%) had a confirmed trial of unclear type, 41 (3.1%) were other, and 221 (16.5%) had no trial-specific language. Among CT Required awards, 730 (69.5%) were pilot, feasibility, or proof-of-concept work. Among 273 awards (20.4%) with a verified sample size, the median was 60 participants (IQR, 44-80), with 236 (86.4%) enrolling [≤]100 participants. Given these findings that CT within NIH K awards are generally small and early-stage, NIH's decision to no longer support trial costs may adversely affect the scientist development pipeline.

BACKGROUND: Federal policy now requires sponsors to submit diversity action plans for clinical trials, yet the modifiable features of trial design associated with enrolled composition remain poorly characterized. English-language proficiency requirements are written into eligibility criteria and recorded as free text in a public registry, making them visible and, in principle, amenable to revision, but their relationship to enrolled composition has been examined only within single disease areas and in small samples of trials. We examined whether these requirements and documented language accommodation were associated with the racial and ethnic composition of enrolled participants. METHODS AND FINDINGS: We conducted a cross-sectional study of interventional trials registered on ClinicalTrials.gov with at least one US site and posted results, using a registry snapshot retrieved in October 2026. The primary cohort comprised the 44,487 trials with exclusively US sites, of which 21,924 (49.3%) reported baseline race and 17,434 (39.2%) reported baseline ethnicity with counts that reconciled with the posted denominator. Both exposures were classified from free-text eligibility criteria using regular expressions validated against independent physician review of 150 trials. For the English-language requirement classifier, sensitivity was 88.2% and specificity was 99.2%. The outcome was the trial-level proportion of enrolled participants in each category, analyzed using generalized linear models adjusted for intervention class, therapeutic area, sponsor, phase, enrollment, sites, start year, US Census region, and the population composition of the trial's site states. An English-language proficiency requirement appeared in 5,736 trials (12.9%); its prevalence rose from 7.4% of trials starting in 2010 to 26.2% in 2023 (odds ratio [OR] per decade 2.91, P<0.001), whereas documented accommodation, an eligibility statement allowing participation in a language other than English (Spanish in 99.9% of flagged trials), remained rare (788 trials, 1.8%). The requirement was associated with lower enrolled proportions of Asian (OR=0.85, 95% CI 0.77-0.93, P=0.001) and Hispanic/Latinx participants (OR=0.85, 95% CI 0.79-0.91, P<0.001), with no evidence of an association for White participants (OR=0.99, 95% CI 0.95-1.04, P=0.75). Documented accommodation was associated with a higher proportion of Hispanic/Latinx participants (OR=3.23, 95% CI 2.82-3.71, P<0.001). Geographic adjustment did not change the statistical significance of the requirement associations and attenuated the accommodation association. The main limitations were the cross-sectional design, which precludes causal inference; the restriction of enrollment analyses to trials reporting baseline demographics, which were more recent than those without such reporting; and possible misclassification of automatically classified exposures. CONCLUSIONS: English-language proficiency requirements became more than three times as common between 2010 and 2023 and were associated with lower enrolled proportions of Asian and Hispanic/Latinx participants. Less common documented language accommodation was associated with a higher enrolled proportion of Hispanic/Latinx participants. Because these criteria are investigator-written and publicly recorded, they may be a tractable target for improving enrollment equity, although whether revising them changes who enrolls requires prospective evaluation

Objective To evaluate the accuracy and cost of RegCheck, an automated large language model (LLM)-based workflow, for identifying clinical trial outcomes and detecting outcome misreporting by comparing its outputs with manual assessments from the COMPare Trials project. Design Validation study. Setting Sixty-two clinical trials originally assessed in the COMPare Trials project, sampled from five high impact general medical journals. Participants Published clinical trial reports and their corresponding prespecified registrations and/or protocols. Main outcome measures Four prespecified research questions were examined. RQ1 assessed outcome extraction recall relative to COMPare. RQ2 assessed accuracy of outcome classification as primary, secondary, or non-prespecified. RQ3 assessed accuracy of misreporting detection relative to COMPare, with additional manual adjudication of discrepancies between RegCheck and COMPare. RQ4 assessed the average per-paper cost of running the automated workflow. Results Across the validation papers, RegCheck achieved 91.2% outcome extraction recall relative to COMPare, and 83.6% outcome classification accuracy. For detection of outcome misreporting, RegCheck's overall accuracy was 85.6%. However, after resolving discrepancies with the original human judgements (which frequently favoured RegCheck's judgement), revised accuracy for outcome misreporting detection was 94.8%. The mean cost of running the workflow was 5.94 USD per paper. Conclusions RegCheck achieved high overall performance with a rigorous manual benchmark for identifying prespecified and reported outcomes in clinical trials, and detecting outcome misreporting, while operating at very low marginal cost. Adjudication of discrepant judgements suggested that RegCheck frequently identified valid issues not captured in the reference standard. Automated outcome checking may offer a scalable way to support editors, peer reviewers, and authors in detecting outcome switching and improving trial reporting.

Background. Long COVID affects an estimated 400 million people worldwide, and is associated with low quality of life. Nearly all completed Long COVID clinical trials reported no benefit, and most required participants to travel to study sites. This requirement systematically excludes severely affected patients. Because there are numerous candidate therapeutics with established safety profiles and regulatory approvals for other indications, scaled, efficient evaluation of therapeutics is needed. Methods. We designed and are conducting a double-blind, placebo-controlled, phase two trial of tirzepatide for Long COVID fatigue, using an entirely remote infrastructure. Design elements included electronic consent, identity and diagnosis verification through document upload, cold-chain delivery of an injectable study drug through a central pharmacy, shared decision-making for dose titration, repeated at-home capillary blood collection in a biospecimen subcohort, weekly participant touch points through study application, wrist-worn wearable monitoring, and clinical support. The trial is operating under FDA Investigational New Drug authorization. Results. This trial enrolled 1,058 participants in 73 days, at least double the rate of any other Long COVID trial. Mean baseline metrics include mean Fatigue Severity Scale of 59.3 (standard deviation [SD] 4.9), daily step count of 3,611 (SD 2,706, general population reference mean 7,731), EQ-5D-5L of 0.6 (SD 0.2), and FUNCAP27 4.0 (SD 1.0), which was a more severely affected population than other clinical trials that collected comparable data. Study processes are working as designed. Participants use existing advocacy and support channels to gather and communicate. Conclusions. A direct-to-participant, siteless infrastructure can support a double-blind placebo-controlled trial of an injectable drug at scale, accelerate accrual, and reach severely affected participants who are routinely excluded by site-based designs. Modernizing drug distribution and regulatory pathways is needed to realize the full potential of decentralized infrastructure for drug repurposing clinical trials.

We propose two new tests to detect drug effects (DE) in trials of one to very few patients followed during two periods (before and after initiation of a treatment). Both methods use longitudinal natural history data to inform the estimation of each patient's DE. The first method uses a non linear mixed effect model (NLMEM) reflecting an expected natural history with a hypothetical drug effect, to estimate the Conditional Distribution of the Drug Effect (CDDE). The second method trains a Pareto Depth Analysis (PDA) algorithm, a machine learning based approach based on outlier detection, that we implement using data simulated under the NLMEM. We evaluated the two tests with a simulation study. We used data from the PROSPAX study in Autosomal Recessive Cerebellar Ataxias (ARCAs, to derive a NLMEM for the Scale for the Assessment and Rating of Ataxia score. The CDDE method provided controlled type I error and, in some scenarios, adequate corrected power, though sensitivity analyses showed vulnerability to misspecification. The PDA method demonstrated lower statistical power except with high score precision. These results highlight different strategies for quantifying treatment effects in ultra rare, patient' specific trials. They can inform methodological design for future ARCA precision therapies.

Background: Five nonprescription statin attempts have failed; a sixth attempt, under the FDA's 2025-issued Additional Condition for Nonprescription Use (ACNU) pathway, is under way. Self-selection evidence is evaluated by whether ineligible consumers are excluded; how often eligible consumers are denied has never been reported. Objective: To quantify denial of access to eligible consumers across the published self-selection record. Methods: Secondary analysis of the three published United States self-selection studies (SELECT, CREST, TACTiC), the FDA guidance, and four advisory committees' questions and votes. We recomputed diagnostic accuracy from published cross-tabulations, derived TACTiC's prespecified unmitigated endpoints, and compared access strategies by decision curve analysis. Results: The guidance's endpoints contain no measure of denial, and every committee question addressing a selection error concerns admission of the ineligible. CREST's sensitivity was 59.0% (95% CI 42.1%-74.4%), 29.5 percentage points below its reported positive predictive value; 41% of clinician-eligible participants were not approved. TACTiC's reference standard reached only application-approved consumers; none of the 10,332 rejected was assessed; sensitivity is inestimable and its prespecified unmitigated endpoints, 84.2% and 80.9% against the published mitigated 90.7% and 98.1%, remain unpublished. Decision curve analysis favors the application over both alternatives across reasonable thresholds at the sample prevalence. Conclusions: For nonprescription statin applications, the self-selection evidence framework counts errors of admission and leaves errors of denial unmeasured; its public health objective, fewer eligible adults untreated, cannot be evaluated on evidence counting only admissions. Four design and reporting changes, none requiring new methodology, would let the next application measure it.

Cash incentives are widely used in clinical research to support recruitment and retention, yet their ethical implications, particularly in high-risk settings such as HIV cure-related studies, remain inadequately understood. This study examined how key stakeholders perceive participant payment relative to other study characteristics when deciding whether to participate in HIV cure research. Using a choice-based conjoint (CBC) experiment, we assessed preferences among people living with HIV (PLWH), HIV researchers, and Institutional Review Board (IRB) members/bioethicists. Participants (N=88) completed eight choice tasks involving hypothetical HIV cure study scenarios varying across six attributes: incentive amount, invasiveness, requirement to pause HIV medications, time commitment, first-in-human status, and cost reimbursement. Across all participants, payment amount emerged as the most influential attribute (29.3%), followed by procedure invasiveness (22.3%) and the requirement to pause HIV medication (17.6%). The highest hypothetical payment level ($200 per visit) was significantly preferred. However, subgroup analyses revealed important differences: PLWH prioritized payment most strongly, whereas researchers and IRB members/bioethicists placed greater emphasis on minimizing invasiveness. The requirement to pause HIV medication was a salient concern across groups, though relatively more influential for IRB members/bioethicists than for PLWH. Time commitment and ancillary cost reimbursement had comparatively lower impact on decision-making. These findings highlight the central role of financial incentives in shaping willingness to participate in HIV cure research, raising ethical considerations regarding potential undue influence, especially in higher-risk studies. At the same time, stakeholders value minimizing procedural risks and disruptions to treatment. This study underscores the need for clearer guidance on appropriate payment practices and for balancing incentives with ethical safeguards, particularly in trials involving antiretroviral therapy interruption.

Background: The clone-censor-weight (CCW) method is increasingly finding application in target trial emulation to compare treatment strategies involving grace periods. However, its performance has not been systematically evaluated against the ground truth. Many studies utilising CCW ignore time-varying covariates and informative pre-existing censoring. Heavy-tailed inverse probability of censoring weights (IPCW) may yield biased estimates and under-coverage. Methods: Simulation Part 1 evaluated the bias, root mean squared error, and coverage probability of CCW analysis under correctly specified and misspecified IPCW models across scenarios with or without time-varying covariates and informative pre-existing censoring. Part 2 varied the confounding strength, assessing whether inference failures could be detected using an IPCW tail-heaviness index (a weighted Hill estimator-derived Pareto-type tail index) and the estimator's standard deviation, the target of bootstrap standard error. Results: Under correct specification, all bias estimates were below 0.005; coverage approached the nominal level of 0.95 with increasing sample size. Omitting the time-varying covariate or mishandling pre-existing censoring yielded bias up to 0.069 and coverage substantially below 0.95. Under strong confounding, confidence intervals failed to attain nominal coverage, even with correct specification and large sample sizes, when the oracle tail-heaviness index was <=1. When the index was >1, the coverage approached 0.95 as the estimator's standard deviation decreased. Conclusions: CCW analysis yields accurate estimates and confidence intervals with nominal coverage when implemented correctly, and confounding is not excessively strong. Investigators should account for time-varying covariates and pre-existing censoring and evaluate the tail behaviour of IPCW distribution and estimator's bootstrap standard error.

Biomedical Machine Learning (ML) is increasingly evaluated through Independent and Identically Distributed (IID) benchmarks, even when deployment involves irregular observation, missingness, distribution shift, and high failure costs. This position paper argues for a stability-oriented reporting standard for safety-relevant biomedical ML. The central requirement is not a specific model class or optimizer, but auditable evidence: performance should be reported under clinically plausible perturbations such as thinning, timestamp jitter, bursty missingness, channel dropout, and Signal-to-Noise Ratio (SNR) shift. When authors invoke training-stability diagnostics, those diagnostics should be hazard-linked: their assumptions, operational meaning, and relationship to stress-test degradation should be stated explicitly. We use controlled mechanism probes to illustrate two hazards that IID scores can hide: observation-process dependence under irregular sampling, and perturbation sensitivity associated with high-curvature training regimes such as Edge of Stability (EoS). We do not propose Neural ODEs, continuous-time models, Sharpness-Aware Minimization (SAM), or EoS metrics as default prescriptions. Instead, we propose a minimum reporting standard that makes robustness claims testable and supports escalation to more complex modelling or optimization only when stress tests justify the cost.

Background: Healthcare contributes 5% of global carbon emissions, with academic and industry-sponsored clinical trials forming a meaningful share. Existing trial emission frameworks inadequately capture data storage and analysis emissions, leaving their contribution to overall trial carbon footprint poorly understood. Objective: (i) Map the flow of trial data from the investigator site to the final trial documentation and study report in the trial master files (TMF), and identify emissions hotspots (ii) estimate emissions from the hotspots and assess whether they materially impact overall clinical trial emissions. Design: A top-down assessment of enterprise-level trial data volumes and associated emissions, and trial-level data-related emissions analysis of two representative trials. Results: The data flow mapping highlighted three carbon emission hotspots: (a) data analysis in a statistical environment (e.g., entimICE) (b) Trial Master File (TMF) storage, and (c) short-term and long-term data storage by external clinical research organisations (CROs). Within entimICE, the total volume of trial data stored for analysis across all active and recently completed trials at AZ was 100-125TB, stored across four servers in Sweden, generating 80 to 100 tonnes CO2eq annually. TMF storage emissions fell below measurable thresholds. CROs stored substantial data volumes, but per-trial emissions were likely insignificant due to economies of scale in large data centres. Conclusion: To our knowledge, this is the first study to comprehensively assess the carbon footprint of industry-sponsored clinical trial data storage and analysis. The study highlighted the complex network of nodes and junctions involved in managing trial data. The results suggest that carbon emissions from trial data storage and processing are currently a small proportion and unlikely to materially impact overall trial-related emissions. Future research should confirm these results with clinical trials that employ emerging data-intensive computational operations, such as the integration of artificial intelligence (AI).

Background: Patient-facing AI intake chatbots can enter clinical use without regulatory review, so their errors may first surface with actual patients. Objective: To develop and pilot a remote, voice-based standardized patient (SP) method for evaluating an intake chatbot's history-taking, documentation, and patient-centeredness. Methods: Four primary care cases had Easy and Difficult variants. In Phase 1, five SPs completed 40 encounters with internal medicine residents rating portrayal fidelity. In Phase 2, SPs completed 80 encounters with a ChatGPT 4o chatbot, recorded whether critical elements were asked about and addressed satisfactorily, and rated summaries; four debriefing interviews were analyzed. In Phase 3, two blinded clinicians rated transcripts and summaries from 101 encounters (32 resident-led, 69 chatbot-led). Results: Portrayal fidelity averaged 4.52 of 5. The chatbot asked about 59.3% of critical elements in Easy versus 50.2% in Difficult variants (adjusted OR 1.49; 95% CI, 1.15 to 1.93) and addressed 73.3% versus 64.6% (OR 1.47; 1.12 to 1.93); near-ceiling summary quality did not differ. SPs reported missed information and rushed pacing. Reviewer agreement was moderate to substantial for element judgments (AC1 0.49 to 0.70) but poor for totals (ICC 0.10 and 0.21). In exploratory comparisons, residents asked about more elements (adjusted difference 12.5 percentage points; 95% CI, 7.0 to 18.1) and satisfactorily addressed more (9.2 points; 3.8 to 14.6); totals did not differ. Conclusions: SP portrayal was consistent enough for pre-clinical chatbot evaluation, and the method detected lower coverage in harder cases. Documentation ratings alone may miss conversational gaps, which require direct measurement.

Objective: To evaluate Epic IP Insights, an electronic health record integrated generative AI summarizer used across a seven-hospital health system, using a multidimensional assurance framework. Materials and Methods: We developed and validated an agentic hallucination detector that decomposed summaries into atomic content units (ACUs) and verified each against source documentation. We then applied it to remaining summaries to estimate the hallucination rate. We also measured source note utilization, textual and semantic similarity of regenerated summaries, and clinician perceptions through structured evaluations. Results: The EPIC IP Insights generated 706 summaries across 445 encounters and used a mean of 5.8% of available notes. The detector was developed on 30 summaries (2,012 ACUs) and validated on 5 held out summaries (295 ACUs). The detector agreed with physician adjudication on 96.95% ACUs in the validation set. Across 671 remaining summaries containing 40,452 ACUs, the hallucination rate was 11.79% (95% CI, 11.47% to 12.10%). Among 385 regenerated pairs of summaries, 32.7% were textually identical and 37.9% were semantically identical. Clinicians found the tool easy to use (97.8%), while responses were more mixed regarding reliance on the output with little verification (45.9%), expected efficiency gains (45.3%), and frequent use (46.1%). Discussion: Clinicians found the tool easy to use but believed that outputs required verification. Systematic evaluation identified the frequency of hallucinations and limited use of available notes which provide directions for future improvement. Conclusion: Multidimensional assurance frameworks are needed to evaluate the safety, reliability, and consistency of generative AI tools. Keywords generative artificial intelligence; electronic health records; clinical summarization; hallucination detection; AI assurance; large language

Background: Six-month multi-month dispensing (6MMD) of antiretroviral therapy (ART) reduces clinic visit frequency and is associated with improved retention in care. During the COVID-19 pandemic, Zambia offered 6MMD to clients 3 months after ART initiation, rather than 6 months standard requirement. We estimated the effect of early (3<6 months on ART) versus standard (6-12 months) 6MMD enrolment on the rate of treatment interruption. Methods: We emulated a target trial using routinely collected electronic medical records from 12 public health facilities in Zambia. Eligible clients were 15 years and above, initiated ART 01/20-08/22, were WHO stage 1 or 2 at ART initiation, and had more than 21 months of potential follow-up. Treatment interruption was defined as missing a scheduled clinic or pharmacy visit by more than 28 days. We applied a clone-censor-weight approach to reduce immortal time bias. Clones were censored when observed dispensing deviated from their assigned strategy. Inverse probability of censoring weights (IPCW) accounted for informative censoring, while inverse probability of treatment weights (IPTW) balanced measured baseline confounders between strategies. We used weighted pooled logistic regression of person-month data to estimate the odds of treatment interruption between early and standard 6MMD enrollers, including follow-up months to model the monthly baseline risk. Results: A total of 6,142 ART clients met the inclusion criteria. 741 (12.1%) were early 6MMD enrollers, 1,590 (26.1%) standard 6MMD enrollers, and 3,811 (62.0%) eligible clients who never enrolled in 6MMD. During follow-up, 268 treatment interruptions occurred. In the primary analysis, early 6MMD was associated with lower odds of treatment interruption than standard 6MMD OR 0.701 (95% CI 0.51-0.97). The predicted cumulative probability of treatment interruption at 18 months was 6.5% under the early 6MMD strategy and 9.1% under the standard strategy (risk difference: -2.6 percentage points). Conclusions: Enrolment in 6MMD at 3-6 months after ART initiation was associated with lower odds of treatment interruption than standard enrolment at 6-12 months, with a predicted absolute risk difference of -2.6 percentage points at 18 months. We found no evidence that earlier access to 6MMD increases the risk of treatment interruption.

Title and abstract screening is among the most labor-intensive steps of systematic reviews. Jev, a low-cost AI decision model, scores records against eligibility criteria without screening labels. We scored 169,982 records from 128 reviews and, after tuning on development reviews, evaluated 23 held-out reviews once and 31 Cochrane reviews post hoc. Jev outranked the best free ranker in all 23 held-out reviews (mean AUC 0.966 vs 0.831) and came within 0.011 AUC of Claude Opus 5.5 on 1999 records at about 1/700 of its list-price cost (US$0.02/1000 records). Combined with active learning, it raised work saved at 95% recall from 0.659 to 0.806. A fixed score threshold fell below 95% recall in 3 of 28 Cochrane reviews, whereas Jev ranking with statistical stopping (post hoc) reached 95% recall in all 51 reviews, reading 86-88% of records. Label-free ranking with Jev was accurate and inexpensive, but stopping safely required screening decisions.

LLMs are increasingly used to mediate patient communication, yet scalable evaluation of their safety, accuracy, and communication quality remains an open problem. LLM judges have emerged as automated evaluators, but whether they can holistically replicate human expert judgment is unvalidated. Informed consent for clinical trials presents a demanding case for such validation because it requires conveying complex information to lay audiences under ethical and safety constraints. We developed a stakeholder-informed seven-criterion evaluation rubric spanning safety, reliability, and communication quality. Clinician reference ratings showed strong interrater reliability across all criteria. We validated the rubric on the Informed CONsent Benchmark (ICON-Bench) and benchmarked 19 LLM judges across multiple implementation strategies. LLM judges achieved strong clinician agreement for safety screening and factual verification (Spearman {rho} > 0.80) but weaker agreement for communication quality ({rho} < 0.60). Safety-specialized guard models underperformed general-purpose models. Patient advocates rated communication quality lower than both clinicians and LLM judges. These findings support LLM judges for scalable patient communication oversight while demonstrating the need for recalibration to patient-centered evaluation standards.

BackgroundCochrane Risk of Bias 2 (RoB 2) assessment is methodologically demanding and resource intensive. Large language models may assist this process but can produce variable judgments. AimTo develop a rule-constrained LLM system for RoB 2 assessment and evaluate its temporal stability and agreement with human reviewers. MethodsRoB2-Evaluator was developed using GPT-5.2 with fixed RoB 2 guidance and a locked instruction framework. Seven RCTs were used for calibration before version freezing. The frozen system was evaluated on 10 separate RCTs, with repeat assessment after one week and comparison against consensus judgments from two independent human reviewers. ResultsTemporal agreement was 90.0% (weighted {kappa} = 0.93), and agreement with human consensus was also 90.0% (weighted {kappa} = 0.93). Methodological rationales were stable in 94.3% of repeated comparisons. Full AI-human conceptual alignment occurred in 85.7%. ConclusionRoB2-Evaluator showed encouraging post-freeze stability, human agreement, and rationale consistency, supporting larger independent evaluations.